Optical flow algorithms (such as the Lucas-Kanade in Pyramids method), can be used on images to detect movement of pixels when compared to a second image. Optical flow refers to a pattern of apparent motion of objects, surfaces and edges caused by relative motion between an observer and scene. Optical flow can be applied to infer the motion of objects within a scene. Optical flow algorithms such as the Lucas-Kandae method may utilize the following constraint equation for a voxel at location (x,y,z,t) with intensity I(x,y,z,t):I(x,y,z,t)=I(x+δx,y+δy,z+δz,t+δt)Optical flow is designed to run against standard images (pictures, web cams, etc.) and is typically operable only on 2-D images.
It may be desirable to perform optical flow on other types of data such as a depth map. Depth maps are a colorless two-dimensional matrix of values representing distance away from a camera (i.e., there is no intensity information). For example, a depth map produced by a 3D depth camera consists of a matrix of values describing distance from the camera of every pixel in the scene. A common application of a depth map is for natural human input. This application requires a system to be able to track movement of interesting points on the subject (i.e., hands, head, etc.).
Furthermore, even if there were effective methods for performing optical flow on depth map data, occlusion in three-dimensional data raises additional technical challenges. Occlusion refers to one object passing in front of another such as a hand passing in front of a head. Because depth maps are restricted to a single camera's point of view, occlusion is likely and cannot be resolved utilizing alternate point(s) of view.
Thus, methods for performing optical flow on depth map data in order to track objects and/or points with dynamic behavior in those images are necessary.